The technology of capturing information of appearance and motion of an actor who takes dynamic motion has two mainstreams as follows:
One is to use active sensors such as employing laser or patterned light for scanning information of actor's static appearance and capturing joint motions by tracing markers attached on the actor's major body parts where dynamic effects are well observed. Afterwards, a designer manually combines the two types of information to capture dynamic motion of the actor.
Although such a method is commonly used in the field of video contents industry such as making films or advertisements for high-quality visual effects, it requires manual work of many well-trained designers and artists to combine different types of information obtained from various sensors as well as large amount of time to create a model. However, once a three-dimensional (3-D) model of an actor is created, it becomes much easier to edit or make corrections to an appearance or animation of new motions.
Meanwhile, another approach is characterized by capturing an actor's appearance and motion information by means of multi-view images where the actor's dynamic motion is recorded from multiple viewpoints. There exist many different technologies in this method depending on whether or not an initial model is employed and how to initialize the model, application range of multi-view image information, reconstruction priority of appearance and motion, as well as shape deformation method.
Most of recent studies focus on, as a method of reconstructing an actor's appearance and motion information, creating a genuine human body model of the actor from a 3-D scanner or multi-view images of key frames, and deforming an actor model's appearance in a manner that a projection error for each frame is minimized between input multi-view silhouette information obtained by applying the actor model as created above for each frame and the multi-view silhouette information obtained by projecting the actor model onto each camera.
Even when relying on multi-view silhouette information, however, often reported is that the information is not sufficient to represent an actor's appearance who takes a complex motion. Thus, various correction means based on graphical user interface (GUI) are suggested in an effort to deal with such a problem.
Furthermore, the method of creating an actor's genuine human body model by means of a 3-D scanner may suffer imperfect reconstruction of genuine shape deformation characteristics if they have not appeared at an initial actor model. Such a problem is rooted in the limitations of the mesh-based local shape deformation method which is used to handle shape deformation of a human body.
Still further, in case of the method that creates an actor's genuine human body model from multi-view images of key frames, the actor's shape deformation at every motion in the respective frames is reconstructed based on the initial actor model's appearance information, thus extensively dependent on the accuracy of the initial actor model's appearance.